-
-
Notifications
You must be signed in to change notification settings - Fork 256
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[INFRA-1809] Private update center improvement #245
Conversation
Build failed; the context from the latest run is: Expand to view
|
Features not exercised in Jenkins project infra are unlikely to be maintained well. Note that this will not be merged, as it has content specific to your environment, rather than just generalizations (which could be). It seems you should rather maintain a fork with your specific data, rather than seek merging into upstream. Wouldn't https://github.com/yandex-qatools/juseppe be better for your use case? |
I think there is a misunderstanding. I just find I commit some private files. That's no need. I will remove them. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. Some documentation for README would be helpful tho
Okay, just miss this. |
We have the same use case where I work as @LinuxSuRen has described as the motivation for this PR. |
ping @daniel-beck How do you think about this PR? Thanks for your time. |
Build failed; the context from the latest run is: Expand to view
|
Anyone could help me to review this? Thanks. |
I recommend you maintain a fork of this repository if you want to use it on your own infrastructure. It's not a goal of this tool to be generally useful, and if we have to cut corners, or even overhaul the tool, to optimize it for use in the Jenkins project infrastructure, we will. I'll poke Tyler to see whether he disagrees, but I'm not particularly inclined to add features (and the resulting risk of getting bug reports and RFEs I have no intention in working on) that would not be exercised in Jenkins infra. |
@daniel-beck I can understand what you're thinking. And I agree with you that this org (jenkins-infra) is setting up for Jenkins infrastructure. I can maintain this feature. But I believe that there're lots of people have the same demands as me. Anyway, thanks for your viewpoint. |
@LinuxSuRen Thanks for your understanding. I've talked with others in the past and they mentioned their problems in keeping their installations up to date with this repo, and their specific needs (like ones implemented here), so I understand the use case. It's just something we don't really have the capacity to support. As I wrote I will check in with @rtyler before closing this PR in case he offers a different perspective. A real fork (not like GitHub forks, more like Hudson/Jenkins) of this repo for end user update sites, or perhaps an entirely different tool like https://github.com/jenkinsci/juseppe, seems like a much easier way to handle this use. |
@daniel-beck I definitely understand your concerns and share many of them myself. If I look at pipeline-library however, there has been a fair bit of work done to make that library work outside of the Jenkins project's own CI infrastructure. In that case, the iteration on the code for uses not strictly part of the original intent for pipeline-library have proven to be overall beneficial to the usage on ci.jenkins.io. I think if @LinuxSuRen is volunteering to contribute and improve the code in this repository, that is an overall long term benefit to the update center process running for the Jenkins project, even if it may seem unrelated in the short term. Fortunately, we know @LinuxSuRen so I'm not worried about this being a "drive-by contribution." 😸 |
Build failed; the context from the latest run is: Expand to view
|
We have the same issue: For security reasons and building jenkins images on the fly, we are very dependend on it and need an internal mirror. The code itself is not much and it looks very maintainable. What would make it possible to integrate it in this repo? Any official commitment? I have found a few ways of how people are simulating what this tool is doing. Not sure whats more beneficial for jenkins-infra: Some hacky way which is not controlable or an official one :). I will clone it and work with LinuxSuRen branch. @LinuxSuRen tx for it |
I really hope this official one could also support a private environment. At least, this feature would not break any infra I think. And Thanks @Siegfriedk and @rasmusjelsgaard to using this PR. |
Awesome, I was actually in the process of implementing something like this. However I was also thinking of implementing a limit on how many plugins and war files to keep around. It seems a bit silly to offer every version of every plugin inside a closed network. |
I thought about this as well but hadn't time to look into it. I would probably add a filter like "last 5 versions" or "younger then 5 month" For now, with the cache, it works quite well and when you look into the setup / content of all plugins there are not tooo many which are huge. For example, a quick check on one of our older jenkins mirror setups: |
Size was one thing, but also sync time was a concern for me. Lastly i was thinking about reducing the load on the jenkins-ci infrastructure. |
For what it is worth, I have tried giving it a stab. I have forked master and rebased the changes made by @LinuxSuRen. However they had some issues. For example, it was only the plugins that had their url changed in the produced Since the whole concept of a cache server didn't make sense to me, as the In order to clean up some code, I decided to implement a static I totally understand if this is way to much change to accept back upstream. Besides that I decided to implement a It still doesn't clean up your local The above is implemented in my feature/private-update-center branch. I also got a bit fed up with not having propper output. When syncing from scratch, you will have quite a big period of time with nothing happening on the screen, as it is downloading large ,war files. I added a Progress Bar as a This needs some more attention, as most of the original The progress bar and logging is implemented ontop of the other branch in my feature/better-logging branch. Note that I may still rebase those branches at will when I find bugs or find stuff that I forgot, so don't rely on them 100%. Comments are more than welcome. |
@reenberg Thanks for sharing your ideas here. There're actually some issues with this PR. But I will continue to fix them. |
please view https://www.oschina.net/news/111266/jenkins-plugin-mirror to solve plugins download slowly |
There's no relationship between this repo and mirror. Please visit here to get more help. And thanks for your feedback. |
FWIW this is still on my backlog to review and test in depth; unfortunately I have not yet found the time to do that. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks OK overall. Sorry it took me so long. I think this should be mergeable after the findings below are addressed.
Co-Authored-By: Daniel Beck <1831569+daniel-beck@users.noreply.github.com>
Co-Authored-By: Daniel Beck <1831569+daniel-beck@users.noreply.github.com>
Co-Authored-By: Daniel Beck <1831569+daniel-beck@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Requesting that previous feedback is addressed.
I see it. Sorry, I'll do it as soon as possible. |
@LinuxSuRen No worries, just making sure this doesn't get stalled any longer 👍 |
Sure. Totally understand. And thanks for the reviewing work. |
Co-Authored-By: Daniel Beck <1831569+daniel-beck@users.noreply.github.com>
Co-Authored-By: Daniel Beck <1831569+daniel-beck@users.noreply.github.com>
@daniel-beck I've finished my changes. |
@LinuxSuRen Please try #365. It is a big overhaul of the update center generator internally and with new parameters, but after the latest changes, I think everything you need is included:
While not exactly the same as what this PR does (especially around "cache"), I think this should take care of your needs. Please let me know how this goes. |
@LinuxSuRen I just merged #265 which changes a lot about the update center. For interest to you with this PR:
I think this PR may no longer be needed. Please look at the new version and tell me what you think. |
As a matter of housekeeping, I'm closing this pull request. This isn't a rejection of the proposed changes; just acknowledging that they're based on top of a much older update-center2 and not usable in their current state (plus they might be obsolete). I'd still like to know whether the implemented changes I mention in #245 (comment) / #245 (comment) address your use case, please let me know. If not, I'm open to finding a solution together. |
Good to hear more improvements from here. I'll let you know if there're more ideas from me. |
INFRA-1809